…config update

- Add self-hosted fonts (DM Sans, JetBrains Mono, Space Mono) for consistent rendering
- Refine UI styling across ChatView, Sidebar, WelcomeScreen, MessageBubble,
  DocumentLibrary, SettingsModal, and ConnectionBanner
- Update eval config: default model to claude-sonnet-4-6 with pricing
- Add agent-ui eval benchmark plan

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Welcome page: typewriter effect for title and subtitle with hacker-style
  randomized timing, sequential content reveal, GAIA text pulsating glow
- Feature cards: fixed-height with code hints that get erased by cursor on
  hover, replaced by expanded descriptions typed out hacker-style
- Pixelated red cursor: consistent 8px blocky design with AMD red glow
  across welcome page, chat streaming, typing indicator, and input cursor
- View transitions: smooth crossfade between welcome and chat views
- Agent activity: elegant slide-in/out transitions for tools and thinking
- Chat polish: bouncing typing indicator, scroll button slide, staggered
  chips, input focus glow, smoother message entrance animations
- Global: theme transition CSS, toast exit slide, modal exit keyframes,
  sidebar content fade on collapse, prefers-reduced-motion support

… refinements

- Smooth streaming exit: streaming bubble fades out with content snapshot
  before completed message appears (no duplicate flash or jarring vanish)
- Stop button: AMD red accents for immediate visual priority during streaming
- User messages: removed contradictory left border for cleaner asymmetry
- GAIA avatar: subtle red glow in dark mode ties into accent system
- Copy confirmation: green background tint flash for clearer feedback
- Agent activity: stronger thinking bar glow, visible collapsed summary
- Input area: inset shadow depth, higher placeholder contrast
- Text selection: AMD red tint across entire app for brand cohesion
- Scrollbars: unified 5px themed scrollbars across all panels and modals
- Glassmorphism: consistent backdrop-blur on all floating surfaces
- Button active states: tactile press feedback on all button types
- Hover accents: doc pills, attachments, tool cards use AMD red consistently
- Transition timing: unified to design system variables throughout

…shell whitelist

The suggested "What hardware is in my PC?" query was completely broken due to:
- Missing system info commands (systeminfo, wmic, powershell, lscpu, lspci, etc.)
- LLM defaulting to Linux commands on Windows (no platform awareness in prompt)
- PowerShell pipe commands broken by shlex.split stripping quotes
- Windows /flags (e.g., findstr /i) misidentified as file paths
- Piped commands not validated against whitelist (security gap)

Changes:
- shell_tools.py: Add cross-platform system info commands to whitelist, add
  PowerShell/wmic with read-only cmdlet validation, fix command execution to
  preserve quoting on Windows, add pipe pipeline validation, block dangerous
  shell operators (>, &&, ||, ;), fix Windows flag path detection
- agent.py: Add dynamic platform detection to system prompt so LLM uses the
  correct OS-specific commands (Windows/macOS/Linux)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…URLs

- AnimatedPresence wrapper: delays unmount for CSS exit animations on all
  modals (Documents, File Browser, Settings, Mobile Access)
- Modal exit: overlay fades out + panel slides down (reverse of entrance)
- Session delete: slides left + shrinks + fades (250ms) before removal,
  sessions below smoothly reflow
- Message delete: fades + scales down + shrinks (250ms) before removal
- Session URL routing: sessions linkable via #hash in URL bar, auto-updates
  on session switch with getSessionHash/findSessionByHash utilities

- Default model updated from Qwen3-Coder-30B to Qwen3.5-35B-A3B across
  ChatAgent config, effective model selection, and database defaults
- Added network query guidance: prefer ipconfig, identify primary adapter
  by real Default Gateway, ignore virtual adapters unless asked

Replace count-based session polling with fingerprint comparison that
detects any change (new/deleted sessions, title edits, timestamp
updates). Add guard against empty server responses wiping the sidebar.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Increase Lemonade health check timeout from 3s to 10s and soften the
banner message to acknowledge the server may be busy rather than down.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…hardening, and test plan

Thinking/cursor display:
- Stream LLM reasoning_content as <think> tags through SSE handler
- FlowThought component shows thinking text with red cursor in AgentActivity
- Single cursor rule: only one red cursor visible at any time
- LoadingMessage with sequential red glowing dots while waiting for LLM
- Auto-collapse AgentActivity panel when thinking completes
- Separated thinking events from status events (start_progress -> status type)

Lemonade integration:
- Model badge shows live model from Lemonade health API (not stale session DB)
- Settings modal shows model size, device, context window, GPU, inference speed
- Inference stats (tok/s, TTFT, token counts) on each assistant message
- Model override: custom HuggingFace model with status indicators (found/downloaded/loaded)
- Settings persistence via SQLite settings table

Security hardening:
- Block & operator in shell commands (was only blocking &&)
- Remove foreach-object from safe PS cmdlets (allows .NET code execution)
- Add shlex.split ValueError handling for malformed PS commands
- Improved DANGEROUS_SHELL_OPERATORS regex with word-boundary matching

Agent improvements:
- System prompt trimmed from 25K to 13K chars (removed verbose examples, deduplicated tool refs)
- Enhanced list_indexed_documents with per-doc chunks, sizes, types
- Enhanced rag_status with total index size and document type breakdown
- Better index_document messages (skip/cache/re-index/new)
- Improved read_file error with parent dir context and search_file suggestion
- Friendlier error messages from GAIA's perspective (not technical stack traces)

Test infrastructure:
- Comprehensive 56-case conversational test plan (tests/agent_ui_test_plan.md)
- Test fixture files: CSVs, YAML, Python, empty file for data analysis tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Cursor consolidation:
- ThinkingIndicator in message header types/erases "Thinking..." next to GAIA name
- Cursor only renders when ThinkingIndicator is active (no dual cursor with FlowThought)
- RenderedContent cursor gated on !agentStepsActive (no overlap with thinking cursor)
- Removed dead cursorRef from FlowThought, renamed wasActiveRef2

Message transition fix:
- Skip rendering static DB message during streamEnding phase (return null)
- Removed stream-ending fade/blur/translate animation (caused visible flash)
- Streaming bubble stays in place until unmounted, static message takes over seamlessly

Thinking panel:
- Auto-collapse immediately when thinking completes (no 300ms delay)
- Removed red border from active summary bar
- Removed erase animation from FlowThought (was invisible due to collapse)
- start_progress emits status type instead of thinking (prevents cursors on status lines)

CSS cleanup:
- Consolidated .thinking-dots animation to single global rule in index.css
- Removed duplicate rules from AgentActivity.css and MessageBubble.css
- Removed dead .flow-thought-spinner CSS and reduced-motion override
- Removed dead .loading-message, .thinking-display, .thinking-cursor CSS
- Slower dot animation: 2.4s cycle with ease-in-out for relaxed pulse

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Remove orphaned .msg-entering CSS class (no longer referenced after transition fix)
- Use var(--text-muted) for thinking indicator color (was hardcoded white, invisible in light theme)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The default model was changed from Qwen3-Coder-30B-A3B-Instruct-GGUF
to Qwen3.5-35B-A3B-GGUF in database.py but the test wasn't updated.

The implementation was changed to emit {"type": "status", "message": ...}
instead of {"type": "thinking", "content": ...} but tests weren't updated.

- AgentActivity panel always starts collapsed (thinking text in header instead)
- Summary bar uses stable step count label (no THINKING → 1 STEP text swap)
- Consistent Zap icon always (no spinner → icon swap on transition)
- Removed active/done CSS differences (no padding/font/border/margin changes)
- Immediate auto-collapse when thinking completes (no 300ms delay)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Add AgentEvalRunner (src/gaia/eval/runner.py) that drives multi-turn
  Agent UI conversations via MCP tools and judges each turn with an LLM
- Add scorecard generator (src/gaia/eval/scorecard.py) with weighted scoring
  across correctness, tool selection, context retention, completeness,
  efficiency, personality, and error recovery dimensions
- Add architecture audit (src/gaia/eval/audit.py) for deterministic
  checks (history limits, agent persistence) without LLM calls
- Wire `gaia eval agent` CLI subcommand with --scenario, --category,
  --model, --budget, --timeout, --output-dir, and --backend flags
- Add eval corpus: 12 documents (reports, CSVs, HTML, code, adversarial
  edge cases) with manifest.json for scenario referencing
- Add 5 YAML scenarios covering RAG quality, tool selection, and context
  retention categories with multi-turn conversation scripts and judge criteria
- Add 30+ prompt templates for simulator, judge, and per-scenario runners
- Commit initial eval run results (phase0–phase3 + fix_phase) as baseline
- Strengthen ChatAgent RAG-first prompt: mandatory retrieval before
  answering, anti-re-index guard, response length calibration
- Improve RAG tools, SSE handler, chat helpers, database, sessions, and
  MCP server based on eval findings
- Add unit tests for history limits (tests/unit/chat/ui/test_history_limits.py)
- Update frontend (App.tsx) with eval-driven UI fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…tion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ucination

Added explicit forbidden pattern: after index_document, calling
list_indexed_documents does NOT provide document content — only filenames.
The model was using this as a false "I've checked the index" signal and
then answering from parametric training knowledge instead of querying.

Also added explicit rule forbidding use of training-data knowledge to
answer questions about indexed documents (supply chain, compliance, etc.).

large_document: FAIL 7.3 → PASS 9.6 after fix (was pre-existing FAIL 5.8 at baseline).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Fix DANGEROUS_SHELL_OPERATORS regex to catch trailing > and < edge cases
- Add _BLOCKED_PS_FLAGS set blocking -EncodedCommand, -File, -ExecutionPolicy, etc.
- Add rehype-sanitize alongside rehypeRaw in MessageBubble to prevent XSS
- Unify permission_request handler in ChatView with ALWAYS_ALLOW check and confirm_id
- Fix unbound session_id in _chat_helpers except block (moved before try)
- Add tests/unit/test_shell_guardrails.py with 39 unit tests for shell guardrails

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ragment filter

- PermissionPrompt: add 'Always allow this tool' checkbox with remember state
  so users can suppress future prompts for trusted tools
- sse_handler: apply _TOOL_CALL_JSON_SUB_RE and _THOUGHT_JSON_SUB_RE in
  print_final_answer to strip embedded JSON artifacts from final responses
- sse_handler: fix _TOOL_CALL_JSON_SUB_RE to handle 2 levels of nested braces
  in tool_args (was leaving }}} fragments when args had nested dicts)
- sse_handler: skip flushing end-of-stream buffer content that is only
  whitespace and closing braces (JSON fragment artifacts)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…s format

The model outputs {"thought": "...", "goal": "...", "tool": "...", "tool_args": {...}}
but _TOOL_CALL_JSON_RE only matched JSON starting directly with "tool", causing
the full JSON to be emitted as visible text with a trailing } artifact.

- Extend _TOOL_CALL_JSON_RE with leading .* to match optional thought/goal/plan
  fields before "tool" (common Qwen3 output format)
- Add _json_filtered flag: set True when any JSON block is suppressed, so
  subsequent bare } tokens (structural remnants) are also suppressed
- Strip thought/tool-call JSON from "before" text in think-block state machine
  to prevent pre-<think> JSON from appearing as response content

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ted changes

- Add Qwen3.5-35B-A3B-GGUF to lemonade_client.py MODELS registry and update
  all agent profiles (chat, code, talk, rag, blender, jira, docker, mcp) to
  use it as the primary LLM — fixes the root cause of Qwen3-Coder being loaded
- Update default model in chat/agent.py and ui/database.py to Qwen3.5-35B-A3B-GGUF
- Add settings table to SQLite DB with get_setting/set_setting/get_all_settings
- Add full <think>...</think> state machine in sse_handler.py routing thinking
  content to thinking events instead of discarding
- Enrich platform system prompt with Windows/macOS/Linux shell guidance
- Add richer indexing status messages in rag_tools.py (already_indexed/from_cache/reindexed)
- Update test assertion to match new default model name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ctness, CI triggers, tests

Bugs fixed:
- runner.py: inject `category` from scenario YAML into all result paths
  (scorecard by_category breakdown was always showing "unknown")
- scorecard.py: avg_score now excludes ERRORED/TIMEOUT/BUDGET_EXCEEDED scenarios
  (infra failures with score=0 were diluting the quality average)
- scorecard.py: track timeout and budget_exceeded as separate counters
  (was lumped into "errored", hiding the distinction)
- scorecard.py: remove unused compute_weighted_score() dead code
- audit.py: fix audit_agent_persistence() to check _chat_helpers.py (where
  ChatAgent is instantiated), not routers/chat.py (which never creates it)
- audit.py: tighten audit_tool_results_in_history() check to require messages/
  history + role pattern, not just "tool" appearing anywhere in the file
- runner.py: fix fixer template interpolation to use str.replace() instead of
  .format() — avoids KeyError when fixer.md contains {} in code examples
- runner.py: clean up .progress.json after a successful run

CI + scenarios:
- test_eval.yml: add eval/scenarios/**, eval/corpus/**, eval/prompts/** to
  path triggers so scenario/corpus/prompt changes fire CI
- vlm_graceful_degradation.yaml: replace Windows-only hardcoded path
  (C:/Windows/Web/Wallpaper/...) with a portable corpus-relative path

Tests:
- Add TestAgentEvalScorecard: pass_rate, avg_score exclusion, category grouping,
  summary markdown — all previously untested
- Add TestAgentEvalAudit: return shape, persistence check, tool history check
- Add TestAgentEvalRunner: find_scenarios filters, unique IDs, required fields,
  compare_scorecards regression detection, corpus manifest integrity
- 14 new tests, all passing (21/22 total, 1 pre-existing unrelated failure)

Each document row now shows a folder icon button (on hover) that reveals
the file in the OS file explorer — Explorer /select on Windows, Finder -R
on macOS, xdg-open on Linux. Reuses the existing /files/open backend
endpoint. Buttons are grouped in .doc-row-actions and fade in on hover.

Shows warning banners in the Agent UI when:
- The required model (Qwen3.5-35B-A3B-GGUF) is not yet downloaded
- The loaded model's context window is below the 32768-token minimum

Backend (system.py):
- Extract actual loaded ctx_size from health endpoint all_models_loaded
  (prioritised over catalog default, so --ctx-size overrides are detected)
- Use `is not None` guards so ctx_size=0 correctly triggers a warning
- Case-insensitive model name matching in all_models_loaded loop
- Query /models?show_all=true when no model is loaded to check download status
- Derive lemonade_url from LEMONADE_BASE_URL for dynamic help links

Frontend (ConnectionBanner.tsx):
- Case 3: model not downloaded — links to Lemonade UI + pull command
- Case 4: context window too small — links to Lemonade UI + serve command
- Both cases include "Check again" retry button and are dismissible
- Reset dismissed state when any new warning condition appears

SettingsModal: Context Window row now shows red/green based on sufficiency.

Tests: 9 new unit tests covering safe defaults, URL parsing, insufficient
context, ctx_size=0 edge case, case-insensitive match, catalog failure
graceful degradation, and model download states.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Resolved 12 conflicts between feat/agent-ui-eval-benchmark and main:
- shell_tools.py: drop "config" from SAFE_GIT_COMMANDS (safer)
- _chat_helpers.py: keep `import re as _re` (used for } artifact fix)
- routers/documents.py: use 65536 block size (main — better perf)
- sse_handler.py: keep extended TOOL_CALL_JSON_RE (handles thought/tool prefix), _json_filtered flag, and pre-think JSON stripping (all from branch — streaming artifact fixes); adopt time.monotonic() for timeout (main — correct clock)
- test_shell_guardrails.py: keep `import pytest`
- DocumentLibrary.css: keep doc-row-actions + doc-open-folder styles (Open Folder feature)
- PermissionPrompt.css: keep HEAD's fuller .permission-remember styles
- index.css: take HEAD (no duplicate .beta-badge)
- WelcomeScreen.css: take HEAD (no duplicate terminal CSS); add .welcome-setup-hint from main
- WelcomeScreen.tsx: accept Terminal + useChatStore imports; add notInitialized/noModel hints from main; remove duplicate useEffect blocks (merge artifact)
- SettingsModal.tsx: keep MCP management imports (Plus, Power, Trash2, MCPServerInfo, MCPCatalogEntry)
- MessageBubble.tsx: keep rehypeRaw only (no rehypeSanitize change)

…field

Some Lemonade versions do not include model_loaded at the health response
root level — only all_models_loaded[]. The status endpoint now falls back
to the first non-embedding entry in all_models_loaded when the root field
is absent, so the UI correctly shows the model as loaded instead of
showing the 'model not downloaded' warning banner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Show inference stats (timestamp, latency, tok/s, TTFT, token counts)
subtly on hover for each assistant message. Stats are persisted to the
DB via a new inference_stats column so they survive page reloads.

- database.py: add inference_stats TEXT column with auto-migration;
  update add_message() and get_messages() to persist/load stats
- _chat_helpers.py: fetch Lemonade stats before db.add_message() so
  they are saved with the message
- models.py / utils.py: expose stats as InferenceStatsResponse in the
  messages API response
- MessageBubble: hover-only stats bar showing full timestamp, total
  latency (derived from message timestamps), tok/s, TTFT, token counts
- ChatView: map inference_stats→stats on load; compute latencyMs from
  preceding user message timestamp

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Implements a comprehensive eval framework for testing the GAIA Agent UI
end-to-end: scenario-driven simulation, per-turn LLM judging, scorecard
generation, fix-mode rerun loop, and CI integration.

Key components:
- AgentEvalRunner: drives claude subprocesses per scenario via MCP
- validate_scenario: structural + persona + corpus path validation
- run_scenario_subprocess: score recomputation, PASS/FAIL override guards
- build_scorecard / write_summary_md: metrics with judged_pass_rate
- compare_scorecards: improved/regressed/score_regressed/corpus_changed buckets
- audit.py: trace inspection and trust/distrust tooling

Scenario suite (34 scenarios):
- rag_quality: hallucination resistance, negation, table extraction,
  cross-section queries, budget queries
- context_retention: pronoun resolution, cross-turn file recall,
  multi-doc context, conversation summary
- error_recovery: file not found, empty search fallback, vague requests
- adversarial: large document stress test
- captured: real conversation replays
- real_world: 19 optional scenarios (skipped when corpus absent from disk)

Eval quality fixes (rounds 1-12):
- FAIL score cap moved from data layer to scorecard avg_score computation
  (raw scores preserved in trace files)
- compare_scorecards: graceful skip on missing scenario_id (was KeyError)
- sorted() TypeError fixed when result status is None
- Persona non-string type validation added
- SKIPPED_NO_DOCUMENT status for missing corpus files (excluded from metrics)
- Real-world manifest merged at runtime so eval agent has full ground truth
- BLOCKED_BY_ARCHITECTURE mismatch warning for hallucinated arch blocks
- corpus_changed bucket isolates corpus availability changes from regressions
- 88 unit tests covering all runner/scorecard logic paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Parallelize MCP server connections in MCPClientManager.load_from_config()
  so failing servers don't block each other (was sequential, ~2s per failure)
- Pre-warm LemonadeManager at server startup so first message skips HTTP
  health/models calls
- Add per-session ChatAgent cache to avoid full re-construction on every
  follow-up message (setup drops from ~3s to 0ms on cache hit)
- Evict cached agent when session is deleted
- Fix SSE streaming: yield 'Connecting to LLM...' immediately before
  producer thread starts so browser shows feedback without delay
- Pad SSE events to >=512 bytes to flush Chromium's ReadableStream buffer
  on every event (prevents batch-dump at stream end)
- Keep AgentActivity panel visible after streaming ends so users can
  expand thinking details; remove auto-collapse and thinking-only hide
- Add PERF timing logs to _run_agent() for setup and process_query phases

Add expected_model_loaded field to SystemStatus. The backend checks the
loaded model against the configured default (Qwen3.5-35B-A3B-GGUF) or
the user's custom_model override. The ConnectionBanner shows a new Case 5
warning naming the loaded model, the required model, and a fix command.
When both the wrong model and a small context window are detected, a
combined message is shown since loading the correct model fixes both.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Backend:
- system_status endpoint now sets expected_model_loaded=False when a
  model is loaded that doesn't match the required default (or the user's
  custom_model setting stored in the DB)
- Respects custom_model override so users who configured an alternate
  model don't see false-positive warnings
- LemonadeManager pre-warm at startup uses min_context_size=0 so it
  only checks reachability without triggering unwanted model reloads
- SystemStatus Pydantic model gains expected_model_loaded field

Frontend:
- ConnectionBanner: new Case 5 banner (Cpu icon) shown when the wrong
  model is running — names both the loaded and expected models, links to
  Lemonade UI, and collapses the context-size warning since loading the
  right model fixes both
- ConnectionBanner: tracks expected_model_loaded transitions so the
  banner re-shows if the model changes back to an unexpected one
- SystemStatus TypeScript type gains expected_model_loaded field
- AgentActivity: remove unused hasToolsOrErrors local variable

Tests:
- Four new test cases: wrong model loaded, expected model loaded,
  wrong model + small context, custom_model override respected

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… manifest absent

In CI the real_world corpus manifest is not checked into git, but the
scenario YAML files are. Both cross-reference tests now detect this and
skip real_world scenarios when REAL_WORLD_MANIFEST doesn't exist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Import faiss, sentence_transformers, ChatAgent, RAGSDK, and
MCPClientManager in a background thread during lifespan startup so
first-message lazy imports are already cached in sys.modules.

Also runs in parallel with the existing LemonadeManager pre-warm,
keeping startup time overhead minimal.

…, pylint suppress)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…, pylint suppress)

Remove ChatAgent/RAGSDK/MCPClientManager from startup pre-load — their
import trees pull in gaia.apps.* which instantiate AgentSDK at module
level, triggering LemonadeManager.ensure_ready() and causing Lemonade
to switch to the default 0.6B model on server startup.

Only pre-load faiss and sentence_transformers (pure libraries, no side
effects).